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Abstract. Given a permutation tt G <Sn, construct a graph Grr on the vertex 
set {1, 2, . . . , n} by joining i to j if (i) i < j and 7r(i) < ■n{j) and (ii) there is no 
k such that i < k < j and 7r(i) < 7r(fc) < 7r(j). We say that tt is forest-like if Cr 
is a forest. We first characterize forest-like permutations in terms of pattern 
avoidance, and then by a certain linear map being onto. Thanks to recent 
results of Woo and Yong, this shows that forest-like permutations characterize 
Schubert varieties which are locally factorial. Thus forest-like permutations 
generalize smooth permutations (corresponding to smooth Schubert varieties). 

We compute the generating function of forest-like permutations. As in the 
smooth case, it turns out to be algebraic. We then adapt our method to 
count permutations for which is a tree, or a path, and recover the known 
generating function of smooth permutations. 



1. Introduction 

Take a permutation tt ~ 7r(l)7r(2) • • • 7r(n) in the symmetric group iS„. Let 
be the graph on the vertex set {1,2,..., n} with an edge joining i to j if and only 
if (i) i < j and n{i) < 7r(j) and (ii) there is no i < fc < j with 7r(i) < 7r(fc) < 7r(j). 
An example is shown in Figure We say that tt is forest-like if is a forest 
(i.e., has no cycle). Note that the edges of G^ correspond to the edges of the 
Hasse diagram of the sub-poset of consisting of the points (i,7r(i)) (Figure 
left). This (sub-)poset is known to play a crucial role in the Robinson-Schensted 
correspondence [Tl| . 

Consider also the following construction, borrowed from jJSl- Label n columns 
by l,2,...,n, and place n — 1 vertical dividers between the columns. Draw a 
horizontal bar between column i and column j if and only there is an edge joining 
i and j in Cr- These bars are simply the horizontal projections of the edges of the 
Hasse diagram. 
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Figure 1. The permutation 7r = 6437518 2, the associated 
graph Gtj and the corresponding collection of bars. 
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We use this construction to define a linear map from Z"^^ to If^^^ ^ where e(7r) 
is the number of horizontal bars in the diagram (also the number of edges in Gtt). 
Choose a linear order on the bars, and associate variables ai with the vertical 
dividers and /^^ with the horizontal bars. If the fcth horizontal bar starts in column 
i and goes to column j then we set 



The map L^r sends (ai, . . . , a„_i) to (/3i, . . . , /?e(7r))- In the example above we have 
/3i = ai + q;2 + as, 1^2 = "2 + 0^3, . . . , /^g = ag + oit. 

Our first result describes forest-like permutations in terms of the map L^, and 
gives a characterization of these permutations in terms of pattern avoidance. 

Theorem 1. For tt ^ Sn the following are equivalent: 

(1) the graph G^r is a forest; 

(2) the linear map L,^ : Z"^^ TL'^-'^'^^ is onto; 

(3) the permutation tt avoids the patterns 1324 and 21354. 

We need to clarify the third point. A permutation tt avoids the pattern 1324 
if one cannot find indices p < q < r < s such that n{p) < 7r(r) < 7r(g) < 7r(s). 
Similarly, tt avoids the pattern 21354 if every occurrence of the pattern 2154 is a 
subsequence of an occurrence of 21354. That is to say, for all indices p < q < r < s 
such that 7r((jf) < 7r(p) < 7r(s) < 7r(r), there exists a t such that q < t < r and 
■jt{p) < n{t) < 7r(s). The notation was introduced by J. West in his thesis [2l], and 
appears, for instance, in Jl]. There are several equivalent ways to describe the 
latter avoidance condition. In particular, it is easy to see that, in the terminology 
introduced by Woo and Yong jJH]; avoiding 21354 is equivalent to avoiding 2143 
with Bruhat condition (1 ^4). However, the first description is more symmetric, 
more clearly showing that tt avoids 21354 if and only if tt"^ does. 

Given that a linear map Z"~^ is bijective if and only if it is onto and 

e = n — 1, we obtain the following result. 

Corollary 2. The map L^r is a bijection if and only if Gtt is a tree. In this case 
we say that tt is tree-like. 

Our second result is the enumeration of forest-like permutations. We will show 
that their generating function is 



We also enumerate several natural subclasses of forest-like permutations, such as 
tree-like permutations. 

The original motivation for studying tree- like permutations came from a question 
of Woo and Yong related to Schubert varieties. It is known that Schubert varieties 
can be indexed by permutations ^21) and various properties of Schubert varieties 
have been translated into properties of permutations. One famous example is that 
a variety is smooth if and only if the associated permutation avoids the patterns 
1324 and 2143 pBl- A weakening of smoothness is the locally factorial property, an 
algebra-geometric condition which states that all local rings are unique factorization 
domains. Woo and Yong established a condition for being locally factorial which is 
equivalent to being onto [231 Prop. 2]. They conjectured that this holds if and 
only if TT is 1324 and 21354 avoiding [22. Theorem ^ settles this conjecture. 




(1) 



I-. 



F{x) 



(1 - x){l -4x- 2x^) - (1 - 5a;)Vl -4x 
2(1 -hx + 2a;2 - x^) 



(2) 
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We note that every smooth permutation (1324 and 2143 avoiding) is forest-like. 
Smooth permutations have been counted before ^^I; and their generating function 
is: 

l-5a; + 4a;^+ x\/l -Ax 
~ ^ l-6x + 8a;2-4a;3 ' 
As Reference is not easily available, we will show how to adapt our proof 
of (jSJ to enumerate smooth permutations. The series S{x) occurs in several other 
enumeration problems [Jj. 

Remark. Results of Cortez QHl) and independently Manivel ^21) show that 1324 
and 21354 avoidance is necessary and sufHcient to characterize which Schubert 
varieties are generically locally factorial. Here generic has the following sense: the 
variety is smooth at almost all points but has a closed subset 1^ where it is not 
smooth, and in that closed subset it is factorial at almost all points. 

We will proceed as follows. In Section|2lwe prove Theorem^] The proof involves 
a fourth condition, equivalent to those of Theorem which uses a certain sorting 
procedure on the bars. In Section^lwe count forest-like permutations and several of 
their natural subclasses, such as tree-like permutations and smooth permutations. 
We conclude in Section by describing several simple bijections related to some of 
our enumerative results, and state some open problems. 

2. Characterization of forest-like permutations 

The aim of this section is to prove Theorem We begin with proving that 
(1) (3) and (2) =J> (3) by proving the contrapositive: if tt contains 1324 or 21354, 
then Gtt contains a cycle and L^r is not onto. 

2.1. Permutations containing 1324 or 21354 

We first look at the structure found in the diagrams of permutations containing 
1324 and 21354. We begin with a very simple lemma which follows from the defini- 
tion of the diagram of bars (alternatively, from the definition of the Hasse diagram 
of a poset). 

Lemma 3. Let n e 5„. If p < q and it{p) < TT{q) then there is a sequence 
p = Po < Pi < ■ ■ ■ < Pk = q such that Tr{pi) < 7r(pi+i) and in the diagram for tt 
there are horizontal bars from column pi to column Pi+i for each i = 0, . . . , /c — 1. 

Lemma 4. Given a permutation tt, 

(a) ifiT contains the pattern 1324 then there are indices p < q < r < s such that 
tt(j)) < 7r(r) < TT{q) < tt{s) and in the diagram for tt there are horizontal 
bars from p to r and from q to s. 

(b) if TT contains the pattern 21354 then there are indices p < q < r < s 
such that Tr{q) < 7r(p) < tt{s) < 7r(r) and in the diagram for tt there are 
horizontal bars from p to s and from q to r. 

Proof. The general idea is the following. If we have an occurrence of the pattern 
that does not satisfy the requisite bar conditions, then we find a tighter occurrence 
that satisfies them. 

For instance, start from an occurrence of the pattern 1324, that is, from a se- 
quence p < q < r < s such that 7r(p) < 7r(r) < ^{q) < 7r(s). Define p' := 
max{i < q : 7r(i) < 7r(r)} and r' := Tam{j > q : 7r(r) > 7r(j) > 7r(p')}. Then 
p < p' < q < r' < r < s, the sequence p', q, r', s corresponds to another occurrence 
of 1324, and there is a bar between columns p' and r'. 

The rest of the lemma is proved by similar arguments. □ 
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Remark. Point (b) in the above lemma shows that 21354 avoidance can also be 
described graphically as follows. Take a permutation tt and plot it as on the left of 
Figure Represent by straight lines the edges of the Hasse diagram of the poset 
{{i,TT{i))}. We thus obtain the natural embedding of G^. Then tt avoids 21354 if 
and only if this embedding of G^r is planar (no edges cross). This does not mean 
that avoiding 21354 is necessary for G^r to be planar: for instance, the permutation 
TT = 2143 contains 21354 but G^r is planar (though its natural embedding is not). 

Lemma 01 is illustrated in Figure [2 where the solid lines indicate a single bar 
and the dashed lines indicate a sequence of bars (coming from Lemma EJ. 



Case 1324 
p q r s 



Case 21354 
p q r s 



Figure 2. Patterns in the bar diagrams of permutations contain- 
ing 1324 or 21354. 

We now show that the occurrence of either of the two "forbidden" patterns impHes 
the existence of cycles in G^r, and prevents L^r from being onto. First, from Figure[2l 
we can read off cycles in G^. For example in the 1324 case we have a cycle that 
starts at p, goes to r then by a sequence of edges goes to s then to q and finally 
by another sequence of edges we return to p. This is a true cycle, as it contains 
the edge joining p to r only once. Similarly, in the 21354 case, there is a true cycle 
visiting p, s, q, r in this order. Secondly, we also see that there are nontrivial linear 
dependencies among the /Sj. In the 21354 case the sum of the solid bars equals 
the sum of the dashed bars, and a similar event happens in the 1324 case. This 
prevents G^ from being onto. 

So if the permutation contains 1324 or 21354 then G^r has cycles and is not 
onto. Taking the contrapositive gives (1)=>(3) and (2)=>(3) in Theorem^] 

2.2. Sorting the horizontal bars 

In this subsection, we define a new condition (2') that clearly implies the surjec- 
tiveness condition (2). We then prove that (2') is implied by the pattern avoidance 
condition (3), and finally that (2') implies the acyclicity condition (1). Combined 
with Section ITTl this proves that the four conditions (1), (2), (2') and (3) are 
equivalent, and establishes Theorem The structure of the proof is schematized 
below. 

Section 12.11 

^ CL) Gtt is a forestj ^ Qi) tt avoids both patternsj 



Lemma 




(^2) is onto^ 



Lemma 



Lemma El 
(^2') the bars are sortable^ 



In the construction of the diagram for a permutation we placed no condition on 
the ordering of the horizontal bars from top to bottom. We now describe a way 
to attempt to sort them. Create a second diagram with the same columns but no 
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horizontal bars. We now look for bars to move to the second diagram by scanning 
the vertical dividers from left to right, looking for any divider which is intersected 
by exactly one horizontal bar. As soon as we find such an intersection we move the 
corresponding horizontal bar to the second diagram and put it above any previously 
moved bar. We then repeat this scanning process, starting again from the leftmost 
divider, until no divider intersects exactly one horizontal bar. If at this stage all 
the horizontal bars are moved over, we say that 

(2') the bars are fully sortable. 
By construction, this can only happen when the number of edges satisfies e(7r) < 
n — 1. An example of a fully sorted diagram is shown in Figure El 



ai ag 

Figure 3. The permutation 7r=1675342 and the associated 
sorted diagram of bars. 

Assume the bars are fully sortable. In terms of the equations d, this means 
that at least one new variable Ui occurs in each equation. More precisely, if Vk 
denotes the set of variables ai occurring in Pi,. .. ,13k, then Vk C Vk+i- Hence, 
given 13 € Z'^f'^), the system (0) can be solved for a by backward substitution from 
the top equation to the bottom equation. Consequently, we have the following. 

Lemma 5. // the bars are fully sortable then L^^ is onto. 

In other words, (2') impHes (2). We shall see below that the converse is also true. 
This will be a consequence of Theorem and the following lemma, which proves 
(the contrapositive of) (3)=>(2'). 

Lemma 6. // the bars are not fully sortable then n contains 1324 or 21354. 

Proof. If we stopped before all the bars have been moved over then it must be the 
case that for what remains all the vertical dividers intersect either zero, or two or 
more horizontal bars. We will work with these remaining (i.e., unmoved) horizontal 
bars. 

Suppose that column a is the leftmost column which has the start of a bar, then 
as noted above it must be the start of at least two bars (otherwise we would have 
moved the bar over). Let c denote the column where the longest horizontal bar 
starting in column a ends. Let b be the rightmost column satisfying a < b < c and 
7r(c) < n{b) (such a b exists because the end of a second bar that starts in a satisfies 
both conditions). 

We now consider cases on how to cover the vertical divider to the right of column 
b with a second horizontal bar. 

Case (1). There is a horizontal bar that begins at b. This bar ends at some 
position d, which, by the choice of b, satisfies d > c. In this case we have that 
a < b < c < d while n{a) < 7r(c) < 7r(6) < n{d) and so tt contains the pattern 1324. 

Case (2). There is a horizontal bar that begins at column d where d < b and 
crosses to some column e where e > b. By the choice of b, we have a < d. Since d 
lies between a and c we must have that Tr{d) < 7r(a) or 7r(d) > 7r(c) (if 7r(d) were 




6 



MIREILLE BOUSQUET-MELOU AND STEVEN BUTLER 



in the interval [7r(a), 7r(c)], there would not be a bar from a to c). So we consider 
subcases. 

Case (2a). If iT{d) < ^{a) then we have that a < d < b < c and Tr{d) < 7r(a) < 
7r(c) < 7r(6) and since there is a horizontal bar from a to c, n contains the pattern 
21354. (Note this includes the possibility that c = e.) 

Case [2bi). Suppose that not only TT{d) > n{c), but also Tr{d) > 7r(fe). Then we 
note that we have a < d < b < e and 7r(a) < 7r(6) < iT{d) < 7r(e) and so tt contains 
the pattern 1324. 

Case [2bii). Suppose finally that 7r(c) < 7r(d) < 7r(6). By the choice of &, we 
must have e > c. Then we note that we have a < d < c < e and 7r(a) < 7r(c) < 
■K{d) < 7r(e) and so tt contains the pattern 1324. □ 

Our final lemma proves that (2')=^(1). 

Lemma 7. // the bars are fully sortable, then G^r is a forest. 

Proof. Suppose on the contrary that Gtt contains a cycle and we can fully sort the 
bars. Now consider the set B of bars that correspond to the edges of a cycle in 
Gtt. At some stage in the sorting procedure, a first bar & of ;S is moved over. At 
this stage, it is the only bar that crosses some vertical divider, say, the ith one. In 
particular, all the other bars involved in the cycle He entirely to the right or entirely 
to the left of the ith divider. In terms of Gjr, this means that removing the edge 
corresponding to b has disconnected the cycle. This is of course impossible, so G^ 
cannot contain a cycle. □ 

3. Generating functions for forest-like permutations 

We now want to prove the enumerative result (pj. At the heart of this result 
is a recursive description of forest-like permutations, given in Proposition El This 
decomposition is then translated into a functional equation defining the generating 
function of forest-Hke permutations (Proposition El, which we solve using the 
kernel method. 

The same decomposition can be recycled to count various subclasses of forest-Hke 
permutations. We will thus also obtain the generating functions of 

(1) tree-like permutations, 

(2) rooted tree-like permutations (the term rooted meaning that 7r(l) = 1), 

(3) path-like permutations (Gtt is a path), 

(4) smooth permutations (tt avoids 1324 and 2143). 

Note that every forest-Hke permutation satisfying 7r(l) = 1 is actually tree-Hke 
(every vertex of G^r is connected to the vertex 1), and thus is a rooted tree- like 
permutation. Note also the following inclusions: 



For n > 1, we denote by /„ (resp. tn,rn,Pn, Sn) the number of permutations 
TT S 5„ of the above five types. We introduce the corresponding generating functions 
F{x) (resp. T{x), R{x), P{x), S{x)). In particular, 



(Rooted tree-Hke^ 



(Path-like) 




(Forest-like) 



a;" ^x + 2x^ + 6x^ + 22a;* + 89x^ + 379x^ + 1661a:^ + • • • . 



n>l 



Our enumerative results are summarized in the following theorem. 
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Theorem 8. The five generating functions defined above are given by: 

(1 - x){l - 4a: - 2x^) - (1 - 5x)Vl - 4x 



Fix) 

T{x) 

R{x) 
Fix) 



2(l-5x + 2a;2-a;3) 
1 - 3a; - 6x2 - (1 - 5a;)^i _ 4-^ 
2(2 - 9a;) ' 



1 - VI -4a; 
2 ' 
l-2x + 2a;2 
^ (l-x)(l-2x)' 



_ 1 - 5a; + 4a;2 +x\/l ~Ax 
^""^ " ^ l-6x + 8x2-4x3 ■ 

From these generating functions it can be shown that there exists positive con- 
stants K such that 

/„~K/(4.61...r, t„~A^t(4.5r, r„ = ip""f) ^K,4"-in-3/2, 

n \ n — 1 / 

p„ = 2"-i-l for 71 > 2, s„^ At, (4.38...)", 

where the growth constants occurring in the asymptotics of /„ and s„ are respec- 
tively the real roots of the polynomial _ 5^2 _|_ 2^ _ 1 and — 6t^ + 8t — A. 

We note that r„ is the (n— l)st Catalan number and has numerous combinatorial 
interpretations [201 Chap. 6]. We give in Section li. 1.21 a, bijective proof of this result, 
as well as another bijection explaining why the numbers p„ are so simple. The terms 
tn have also occurred before and enumerate the number of stacked directed animals 
on a triangular lattice jH]- No direct bijection between stacked directed animals and 
tree-like permutations is currently known. 

The form of our decomposition of forest-like permutations will force us to take 
into account an additional statistic, namely the number of rZ-minima for forest-like 
or tree- like permutations, and the length of the final ascent in smooth permutations. 
This is why we actually obtain bivariate generating functions that refine the above 
theorem (see itTfijl . ifTfijl . ifTzjl ). Other statistics, like the number of descents, could 
also be carried through our calculations. 

3.1. Decomposing forest-like permutations 

If TT e 5„ we say that tt has length n, and write \tt\ = n. We say that Tr{i) is an rl- 
minimum (right-to-left-minimum) if for all j > z, we have > 7r(i). We denote 
by m(7r) the number of r/-minima of tt. Finally, tt is increasing if tt = 12 • • • n. 

Let TT e S„ be forest-Hke. We decompose tt by considering which element maps 
to 1. So suppose that i = tt~^(1) then there are two cases: 

• First case: i = n = \tt\. Then the permutation r G Sn-i defined by r(i) = 7r(i) — 1 
is forest-like. Conversely, starting with a forest-like permutation r G iS„_i we can 
construct a forest-Hke permutation tt S 5„ by letting 7r(i) = r(i) + l for 1 < i < n~l 
and 7r(n) = 1. Note that tt is tree- like if and only if n = 1. 

• Second case: i ~ 7r^^(l) < n. We now focus on this case, illustrated in Figure^ 
Let 

h = min {{TT{i + 1)} U {TT{j) : j < i}). (3) 

So h is the smaller of the lowest value of tt to the left of i or the value of tt at z + 1 . 

First note that for all j > i + 1 we have 7r(j) < h or 7r(j) > 7r(i + 1). If not, 
then for some j we have 7r(i) < h < 7r(j) < 7r(i -I- 1) and 7r~^(/i) <i<i + l<j, 
so the permutation contains the pattern 21354, and cannot be forest-like. Further, 
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if j, > i + I with 7r(j) > 7r(i + 1) and 7r(fc) < ft, then j < k. If not, then 
i<i + l<k<i and 7r(i) < 7r(fc) < 7r(i + 1) < 7r(j), so the permutation contains 
the pattern 1324, and cannot be forest-like. 

The latter property implies that the last h — 2 terms of tt = 7r(l)7r(2) • • • 7r(n) 
are 2, 3, . . . , /i — 1 in some order. Let t be the permutation obtained from tt by 
retaining only its /i — 1 smallest entries, i.e., 

T — 1 7r(n — /i + 3) 7r(n — ft + 4) • • • 7r(n). 

Then t is rooted and tree-like. Similarly, let a be the permutation obtained by 
deleting these ft — 1 smallest entries and subtracting ft — 1 from the remaining 
entries: 

cr = (7r(l) - ft + l) • • • (7r(i - 1) - ft + l) (7r(i + 1) - ft + l) • • • (7r(n - ft + 2) - ft + l) . 

Then a is forest-like. Moreover, a{i) is an rZ-minimum of a. If fT(i) is the fcth 
rZ-minimum of a (read from right to left), define $(7r) = (r, cr, fc). Observe that 
k = m[(j) if = 1 (that is to say, ft = 7r(i + 1)), and fc = 1 if z = |cr|. 



7r(l) 




















'*7r(i+ 1) 








• I 


u 





7r(j) = 1 



Figure 4. The structure of a forest-like permutation. The shaded 
areas show which regions of the embedding in may contain 
points. 

Conversely, starting from a 3-tuple (r, a, k) such that r is tree-like, a is forest- 
like and k < m(a), we can construct a (unique) forest-like permutation tt satisfying 
$(7r) = (r, (T, k). If |t| = ft — 1 and the fcth r/-minimum of a is cr{i), this is done 
by adding ft — 1 to the entries of a, inserting 1 at position i and adding the other 
entries of r to the right of a, in the same order as in r. By looking at the number 
of r/-minima of the resulting permutation tt, we obtain the following result. 

Proposition 9. The map ^ is a bijection between forest-like permutations n with 
TT~^{1) < |7r| and 3-tuples {T,a,k) such that r is rooted tree-like, a is forest-like, 
and 1 < k < m{a). Moreover, 

IttI = |t| + \<j\ and m{n) = ( ^ |J '{l^^- W 

I m[T) otherwise. 

In order to count the various sub-classes of forest-like permutations we have 
defined, we need the following result. 

Proposition 10. Let it ~ (f>(r, cr, fc) be a forest-like permutation such that Tr^^{l) < 
\tt\. Then 

(1) TT is tree-like if and only if a is tree-like, 

(2) TT is rooted tree-like if and only if a is rooted tree-like and k = m{a), 
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(3) TT is path-like if and only if t is increasing, a is path-like and its kth rl- 
minimum a{i) is such that i has degree 1 in Ga, 

(4) TT is smooth if and only if a is smooth and either k = m{a) or k < a{a), 
where a{a) is the length of the final ascent of a: if \a\ = £, 

a{a) = max{i : cr(£ - i + 1) < • • • < cr(£ - 1) < cr(^)}. (5) 

Moreover, 

«/ r = 1 and k < a{(j), 
I \ _ I > if T = ^ o,nd k = m(cr) > a(cr), , > 

1 _ 1 if T ^ I is increasing, ^ ' 




oth 



erwise. 



Proof. The first three results simply follow from the decomposition of Figure 01 
The reader should look at Figures and |2| to see this decomposition specialized to 
the rooted case and the path case, respectively. 

Now let's assume that tt is smooth. Since cr and r are obtained by deleting entries 
from TT, they are smooth as well. This does not restrict the choice of r, since every 
rooted tree-like permutation is smooth. Conversely, when we construct <I>(t, cr, k) 
(assuming that a is smooth and r rooted) we do not create any occurrence of 2143 
if we insert 1 just before the smallest entry of cr. This corresponds to the case 
k = m(cr). 

However, if /c < m{a), then the value h defined by (O satisfies h < 7r(i + 1), 
and the final permutation tt contains 2143 if, and only if, there is a descent in cr 
somewhere to the right of cr(i). In other words, if /c < m(cr), then tt avoids 2143 if 
and only if 1 is inserted in the final ascent of cr, that is to say, k < a(cr). 

A case study finally provides the value of a(7r). □ 



3.2. Functional equations 

We now translate Propositions El and EH into enumerative terms. We first note 
that every pair (r, cr) can be combined in m(cr) different ways. To account for 
this we refine our generating functions by further distinguishing by the number of 
rZ-minima. So let 

n/>l i 

where fn.e. is the number of forest-like permutations of 5„ having I r^-minima. Note 
that F[x) = .?^(1). Define similarly the bivariate series T{x,u), TZ{x,u), V{x,u). 
The case of smooth permutation is a bit different: here, the crucial parameter is 
the length of the final ascent, defined by We thus use a new indeterminate v 
and define 

S{v)=S{x,v)^ J2 s„,,xV = ^5,(x)z;^ 
7i,e>i i 

where Sn/ is the number of smooth permutations of 5„ having a final ascent of 
length £. We define similarly the series TZ{x, v) that counts rooted tree-like permu- 
tations by the same statistics. 
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Proposition 11. The (bivariate) generating functions J^{u) , T{u), TZ{u) andV{u) 
satisfy: 

Tiu) = xu + xuTil) + ^^2 -^(^) - -^(1) _^ (n{u) - xu)P{l), 

u — 1 

T{u) =xu + a;^2 ^(^) + ijiiu) - xu)T'{l), 

u — 1 

Tl{u) ^ xu + xuR{u) + — xu)TZ{l), 

(1 — x)"^ 1 — x 

where = ^{x, 1) and similarly for Moreover, 

For the smooth case, 

S{v) = xv{l -x) + xS{v) + xv{l - x)-"'^^"^ " ^^^^ 



nv) - ^^r^ ) ((1 - ^)is'{i) + 5(1)) - X) 



where 



v-1 

( 

1 



n{v) = ^ + xn{v) + n{v) — ^ 7^ i 

1 — XV \ 1 — XV / 



Proof. We first consider T(u). The terms xu + xuT{\) count forest-like permu- 
tations with 7r~^(l) = |7r|, which have only one rZ-minimum. For the remaining 
forest-like permutations we use Proposition El The generating function of permu- 
tations (T such that T = 1 is: 

Y^T,{x)Y^xu^+^ = xu'Y^T.ix)^^-—^ = ^^2 -^W--^(l) ^ 
e k=i f " " 

while for the permutations such that t 7^ 1 we obtain: 

£ 

'^J'i{x)^{n{u) - xu) (7^(^t) - xu)j^'(l). 

Combining all cases gives the result for T{u). 

The equation for T(u) is proved in a similar way (note that there is no coun- 
terpart to the term xuT[\) since this corresponds to forests where 1 is an isolated 
vertex) . 

For rooted tree-like permutations there is no choice in the way we merge r and 
a and so we obtain a significantly simpler equation (see Figure 

The equation we have obtained for J^{u) shows that the indeterminate u is needed 
to exploit the decomposition of Proposition El This is not the case for path-like 
permutations, and this is why we will not take into account the number of rl- 
minima. If cr is path-Hke, the graph Ga has exactly 2 vertices of degree 1, unless 
cr = 1. If cr is increasing, both of these end vertices correspond to r/-minima. 
Otherwise, only the largest one does (Figure O- The term x^/{l — x)^ in the 
equation corresponds to the case where a is increasing and k ~ \a\. The term 
x/{l — x){'P(\) — x) corresponds to the case fc < \a\. 
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The relationship 10 can be explained by noting that a forest-like permutation 
TT is either tree-like, or is obtained by appending a tree-like permutation t to the 
beginning of another forest-like permutation a. More formally, 

TT = (t(1) + h) (t(2) + h)--- {T{k) + h)a{l)(7{2) ■ ■ ■ a{h), 

where r is tree-like and a is forest-like. Note that m(7r) = m{a). In terms of 
generating functions, this gives T{u) = T{u) + T{l)J^{u). 

We now proceed with the smooth case. Let us first determine the generating 
function So{v) counting the smooth permutations tt such that a(7r) = m(7r) (that is 
to say, 1 belongs to the final ascent of tt). This equality certainly holds if 7r~^(l) = 
|7r|. Otherwise, let us write tt = <I>(t, cr, fc). By comparison of ^ and we see 
that a(7r) = m(7r) if and only if t = 1 and k < a{a). Hence 

So{v) ^ xvil + 5(1)) +xY Se{x) y = xv + xv ~ ^^^^ . (8) 

Z — / Z — / tl — 1 

e k=i 

Combining this with it follows that the smooth permutations tt — (f>(r, cr, fc) 
such that T — 1 but fc = m{a) > a{a) are counted by 

x{S{v)~So{v)). (9) 

In the case where t 7^ 1 is increasing, we obtain the series 

^^(5'(l) + 5(l)-5o(l)) (10) 
while in the case where t is not increasing, we find: 

(n{v) - (5'(1) + 5(1) - 5o(l)) . (11) 

\ 1 — XV J 

The series S{v) is the sum of (IsHTTll. This gives the desired functional equation for 
S{v). 

It remains to count rooted tree- like permutations by the length of the final ascent. 
We obtain an equation for TZ{v) by specializing the above study to the rooted case, 
that is to say, to the case where a is rooted and fc = m{(T). The counterparts of the 
terms ijsHTTll are respectively 

T^oiv)^- , X (TZiv) - Uoiv)) , 7^(l) and (n{v)-— 7^(l). 

1 — XV I ^ XV \ 1 — XV J 

The sum of these four terms is Tl{v), and this gives the desired equation. □ 

3.3. Solution of the functional equations 

We are finally going to solve the equations of PropositionElto obtain Theorem|Hl 
Three of them do not raise any difficulty. Namely, the equation defining 7^(1) is 
readily solved, while the equations defining TZ{u) and TZ{v) can be solved by first 
setting M = 1 (or u = 1) to determine the value of these series at u = 1 (or v = 1) 
and then using these preliminary results to compute the full series. In particular, 

_ xu{2 -u~ uy/l - Ax) _ xu 
^""^ " 2{1 -u + xu^) ^ l-w7^(l)• ^ ^ 

The other three equations (defining TZ, T and S) involve divided differences of 
the form 

A{u)-A{l) 
u- 1 

and cannot be solved by setting u — 1. Instead, we will solve them by using the 
kernel method OIH]. Consider for instance the equation for tree-like permutations. 



12 



MIREILLE BOUSQUET-MELOU AND STEVEN BUTLER 



This is a linear equation with one catalytic variable (u) and two additional unknown 
functions (T(l) and T'(l)). However, these two functions are not independent: by 
taking the limit as u goes to 1 in the equation we find 

T(l) = x + nil)T'{l). (13) 

The coefficient of T(u) in the equation defining T{u) is 



1 



u ~ 1 u — 1 



which vanishes for two values of u. One of these values is a formal power series in 

X, 



^ = = Yx • 

Replacing u by J7 in the functional equation gives a second linear relation between 
r(l) and r'(l): 

Q^xu -r{i) + {n{u)-xu)r'{i). (i4) 

One can now solve (I13|l and H14II for T(l) and T'{\), in terms of x,U,TZ{l) and 
71{U). Then the solution can be written as a pair of rational functions of U using: 

- the expression of TZ{U) in terms of x, U and TZ{1) (see lfT2|l ). 

- the fact that 7^(l) = xU, 

- the equation x = {U ~ 1)/C/^. 

Replacing the expressions of T(l) and T'(l) in the original functional equation 
gives an expression for T{u) in terms of u and U, which can be rewritten as 

Tix u) - , J^ + Vni-2V)-uV{l^2V-2V-) 

^^""'"^ " ii-2v){i + v-uvy ^^^> 

where 



V = u-i= 

2x 

We can use similar techniques to find T(u). However, it is easier to use and 
what we have obtained for T to get 

u) = ^y i^ + y?i^-m-nV{l~2V-2V-) 
^ ' ' il~V~2V^-V^){l + V-uV)^ ^ ' 

where V is given above. 

The solution of the equation defining S{u) is similar to what we have done for 
T(u). One possible expression of the bivariate series that counts smooth permuta- 
tions by the length and the length of the final ascent is 

S(x, u) = xu - — - — '- ^ — 17 

^ ' ' {l + V-uV){l-V-V^-V^){l~xu) ^ ^ 

Putting u — 1 into equations Ijl5ll . Ijl6|l and (I17p and simplifying then gives the 
results of Theorem |H1 



4. Final comments and open questions 

We first show that several bijections are underlying the results presented in this 
paper. We then raise a number of questions of an enumerative or graph-theoretic 
nature. 
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4.1. BiJECTIONS 

In what follows, we discuss three objects closely related to the graph G^: first the 
graph itself, second its oriented version G^r (each edge is oriented from the vertex 
with the lower label to the vertex with the higher label), and finally its natural 
embedding in (where the vertex i is placed at position (i,7r(i)) and the edges 
are represented by straight lines, as on the left of Figure QJ. 

4.1.1. The graph G^r. We first note that the map tt i-^ Gtt is injective. That is, 
one can recover tt from G^. To see this, orient G^ to obtain G^. Then, for every 
vertex i in G^, let a{i) be the number of vertices that can be reached from i by 
a directed path. This is the number of j > i such that n{j) > 7r(i), and the 
sequence 7r(l),7r(2), . . . ,TT{n) can be easily reconstructed (in this order) from the 
list (a(l), . . . , a(n)). For instance, if m vertices can be reached from 1 (that is, 
a(l) = m), then it means that 7r(l) = n — m + 1, and so on (we have assumed 
implicitly that |7r| = n). 

As noted at the beginning of the paper, G^ is the Hasse diagram of a certain 
poset P on |n] = {1,2, ... ,n}. The underlying order is natural, meaning that if 
i < j in P, then z < j in N. (We refer to |19l Chap. 3] for generalities on posets.) 
The n\ permutations of 5„ thus provide n\ distinct natural orders on |n]. Not all 
natural orders are obtained in that way: even for n = 3, there are 7 natural orders 
but only 6 permutations, and the poset in which the only relation is 1 < 3 is not 
obtained from any permutation (Figure [HJ. The posets that are actually obtained 
from the construction tt G-^ are, by definition, the natural orders on |n] of 
dimension 2 J2l Exercise 3.10]. 

Some graph properties of G,r easily follow from the construction. For instance, 
Gtt is isomorphic to G^-i (more precisely, Gj^-i is obtained by relabelling the 
vertex i by 7r(i)). The natural embedding of G^^-i is obtained by refiecting the 
embedding of Gtt through the main diagonal. Of course, G^r is triangle free (every 
Hasse diagram is). In particular, e(7r) < [n^/4j (see [2]) and it is easy to construct 
a permutation showing that this bound is tight. The number of edges of G^ can 
also be interpreted in terms of Bruhat order (see Exercise 3.75], [Ol Ch. 2]): it 
is the number of permutations covering (poset-wise) tt in the Bruhat order. 

3 
I 

2 2 3 3 

I \y A 

1 112 

Figure 5. The 6 posets obtained from permutations of length 3. 

4.1.2. Rooted tree-like permutations. Here, we want to show a simple bijection be- 
tween rooted tree-like permutations of size n and plane trees with n—1 edges. This 
explains why such permutations are counted by the Catalan number G„_i. Recall 
that permutations tt avoiding 21354 are exactly those such that the natural embed- 
ding of Gtt is planar (see the remark following Lemma This holds in particular 
for rooted tree-Hke permutations: the embedding of G,r is thus a (rooted) plane 
tree. Then, observe that the decomposition of forest-like permutations illustrated 
in Figure 131 once specialized to rooted permutations, coincides with the standard 



I I 

13 12 12 3 
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n{2) 



tt{1) = 1 

Figure 6. The decomposition of rooted tree-like permutations. 

decomposition of plane trees (a left subtree joined to the root by an edge, and 
another plane tree, see Figure EJ. This means that every plane tree is obtained 
from exactly one rooted tree-like permutation. This is illustrated in Figure [3 for 
permutations of length 4. 

Figure 7. The 5 rooted tree-like permutations of length 4 and the 
corresponding plane trees. 

4.1.3. Path-like permutations. Consider a path-like permutation tt of length at least 
2. The graph Gjr has two vertices of degree 1. Define a word W{n) on the alphabet 
{U, D} by following the path G^r from the vertex of degree 1 with the lowest label to 
the other vertex of degree 1, encoding each edge of this path by a letter U (like up) 
or D (like down) depending on how the labels of the vertices vary along this edge. 
Examples are shown in Figure |H| It turns out that the map is a bijection from 
path-like permutations of length n to words of length n — 1 distinct from D"~^ . In 
particular, this explains why the number of path-like permutations of length n is 

" H B B a « ffl « 

UUU DDU DUU UUD DUD UDD UDU 

Figure 8. The 7 path- like permutations of length 4 and the cor- 
responding words. 

Again, this result follows from the decomposition of path-like permutations that 
led to the equation of Proposition Indeed, this decomposition gives, for the 
noncommutative generating function defined by 

J2 Win) 

7r path-like 

the following equation: 

V^e + U+ + D+U+ + {V - e)DU*, 

where e denotes the empty word and we have used the standard notation D+ = 
£)' and U* — J2i>o is easy to see that the solution of this equation is 

r = {U,D}* -D+. 
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That is to say, the non-empty words W{'!r) are those containing at least one U, and 
each such word corresponds to a unique path-like permutation. 




Figure 9. The decomposition of path-Hke permutations. 



4.2. Open problems 

4.2.1. Enumeration. In this paper, we have characterized and counted forest-like 
permutations and some of their natural subclasses. This work raises similar ques- 
tions for several supersets of forest-like permutations. The most natural ones are 
probably the following two: 

(1) what is the number oi plane permutations of 5„, that is to say, permutations 
avoiding 21354? 

(2) what is the number of permutations associated with a Gorenstein Shubert 
variety? These permutations generalize forest-like permutations, and have 
been characterized in |23) . 

We also recall that the enumeration of 1324 avoiding permutations is still an open 
problem plEl- Permutations avoiding 2143 are called vexillary and are equinumer- 
ous with 1234 avoiding permutations 012], which have been enumerated in [T3| . 

Another natural question is to count permutations n by their length and the 
number e(7r) of bars in their bar diagram (which is the number of permutations 
covering tt in the Bruhat order). To our knowledge, the bivariate series 

n>0 7rG5„ 

is not known. However, the total number of edges in the bar diagrams of permuta- 
tions of Sn is known: if 

then 

e{n) = (71 + iy.{H{n + 1) - 2) + n! 
where H{n) = 1-1-1/2 + -- - + l/n is the nth harmonic number. Indeed, as communi- 
cated to us by David Callan, it is not hard to see that the number of permutations 
of Sn having a bar going from i to j, with i < j, is n\/{j — i + 1), and the above 
result follows easily. 

Note that e{n) is also the number of edges in the Basse diagram of the Bruhat 
order of 5„. The exponential generating function of the numbers e(n) is 

n>0 \ J \ / 
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The average number of bars in a permutation of 5„ is 

C ( 72 ) 

-V = log (n) n+{-2 + 7) n + log (n) + 1/2 + 7 + 0(l/n) 

where 7 is Euler's constant. This can be compared to the average number of non- 
inversions, which is known to be n{n + l)/4. Related questions have recently been 
studied in 

4.2.2. Graph questions. We have seen that the labeled graphs obtained from the 
map TT are the Hasse diagrams of natural orders of dimension 2. One can also 

wonder which unlahelled graphs are obtained through our construction. Clearly, 
these graphs must be triangle free. However, this is not a sufficient condition. 
For example, by an exhaustive computer search one can verify that the triangle- 
free graph formed of the vertices and edges of a cube is not produced from any 
permutation in S^. 

Note that, by Section H .1.21 all unlabelled trees (and thus all unlabelled forests) 
are obtained through our construction. 
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